Community Health Survey Methodology
The Community Health Survey (CHS) has been conducted annually by the New York City Department of Health and Mental Hygiene since 2002. Data collected from the CHS are used to better understand the health and risk behaviors of New Yorkers and to track key indicators.
Target Population
The target population of the CHS includes adults in non-group quarters aged 18 and older who live in New York City and have a cellular telephone or live in a household with a landline telephone. Prior to 2009, the CHS only included those living in households with a landline telephone.
Health Topics
Most years the CHS includes approximately 125 questions, covering the following health topics: general health status and mental health, health care access, cardiovascular health, diabetes, asthma, immunizations, nutrition and physical activity, smoking, HIV, sexual behavior, alcohol consumption, cancer screening and other health topics. A core group of demographic variables are included every year.
Sampling
The CHS uses a stratified random sample to produce neighborhood and citywide estimates. Neighborhoods are defined using the United Hospital Fund's (UHF) neighborhood designation, which assigns neighborhood based on the ZIP code of the respondent. New ZIP codes have been added since the UHF's were originally defined. There are 42 UHF neighborhoods in NYC. However, to avoid small sample sizes for CHS estimates, UHF estimates are generally collapsed into 34 UHFs/groups.
CHS data are now available at the Community District (CD) level. Data from the 2015+2016, 2017+2018, and 2019+2020 CHS were combined and weighted by CD, using detailed geographic information provided by respondents in the survey. We anticipate producing weighted CD estimates and data sets every two years, with the next combination including 2021+2022.
A computer-assisted telephone interviewing (CATI) system is used to collect the survey data. The CHS sampling frame was constructed with a list of telephone numbers provided by a commercial vendor. Upon agreement to participate in the survey, one adult is randomly selected from the household to complete the interview.
Interviewing is conducted in a variety of languages. Every year, the questionnaire is translated from English into Spanish, Russian, and Chinese. In 2018, Bengali and Haitian Creole were added. Typically, data collection begins in March of the study year and ends in December. The average length of the survey is 25 minutes.
Prior to 2011, the Health Department used census 2000 data for weighting. In 2011, the Health Department updated the weighting methodology of the CHS, consistent with other large state and national surveys, to use the census 2010 data. In addition, the Health Department used some new demographic variables, such as number of adults in the household, presence of children in the household, marital status and educational attainment. After analyzing possible effects of these changes, the Health Department found that the updated methodology has minimal or no effect on CHS health estimates and does not impact the interpretation of trends in prevalence (percentages) over time. Full details can be found in the methodology update report (PDF). Beginning in 2012, American Community Survey 2011 data were used for control totals. Every year since 2012, the American Community Survey data from the previous year have been used for control totals, along with census data
Limitations
The survey sampling methodology does not capture the following groups: adults in households without any telephone service, and prior to 2009 adults who could only be reached by cell phone. The CHS also excludes adults living in group quarters, such as college dormitories or nursing facilities.
Sample Size, Response and Cooperation Rates
The sample size (completed interviews), the response rate and the cooperation rate are provided for each year of the survey in the table below.
Response and cooperation rates are measurements of overall survey participation among those sampled. More specifically, the Cooperation Rate is defined as the number of those who participated in the survey, divided by the number of individuals in the sample who were contacted and identified as eligible. The Response Rate is a more conservative measure and is defined as the number who completed the survey, divided by all individuals who are known to be, or are likely to be, eligible.
There are multiple and changing ways to calculate these rates, including ways to combine survey participation rates from landline and cellular telephone samples. The NYC Health Department follows the Standard Definitions published by the American Association for Public Opinion Research (AAPOR), currently in its 9th edition. When describing survey participation rates for the CHS, the Health Department generally reports AAPOR’s Response Rate #3 and Cooperation Rate #3 (see Table). Response Rate #3 assumes that some portion of the numbers with unknown eligibility are in fact ineligible and are therefore removed from the denominator. Information from cases with known eligibility (either known eligible or known ineligible) is used to estimate the percentage of unknown eligible cases that are ineligible.
You can get full disposition reports for 2015-2020 (PDF) and combined landline and cell phone survey participation rates using the most recent AAPOR definitions. You can also get disposition reports for 2009-2018 (PDF) using earlier participation rate formulas, as well as full disposition reports for 2004-2008 (PDF).
| Community Health Survey Participation Rates | |||
|---|---|---|---|
| CHS survey participation rates are based on Standard Definitions established by the American Association for Public Opinion Research (AAPOR). |
|||
| Year | Analytic Sample | Response Rate (AAPOR#3) | Cooperation Rate (AAPOR#3) |
| 2002 | 9,674 | 36.0% | 69.0% |
| 2003 | 9,802 | 44.2% | 63.3% |
| 2004 | 9,585 | 29.0% | 62.5% |
| 2005 | 9,818 | 38.7% | 79.3% |
| 2006 | 9,714 | 35.8% | 90.7% |
| 2007 | 9,554 | 32.8% | 90.4% |
| 2008 | 7,554 | 33.3% | 80.7% |
| Starting in 2009, cell phone interviews were added. Landline and cell sample rates were combined per AAPOR Cell Phone Task Force Report (2010). | |||
| 2009 | 9,934 | 37.7% | 89.4% |
| 2010 | 8,665 | 39.0% | 89.4% |
| 2011 | 8,792 | 40.0% | 89.1% |
| 2012 | 8,797 | 39.1% | 86.6% |
| 2013 | 8,698 | 39.5% | 88.7% |
| 2014 | 8,562 | 40.5% | 88.9% |
| Rates revised per AAPOR Standard Definitions 9th edition (2016). Unrevised rates shown in parentheses for comparison. | |||
| 2015 | 10,172 | 17.4% (44.3%) | 84.2% (89.6%) |
| 2016 | 10,000 | 17.3% (42.5%) | 85.3% (90.2%) |
| 2017 | 10,005 | 13.7% (39.0%) | 85.6% (90.2%) |
| 2018 | 10,076 | 8.4% (29.8%) | 82.8% (88.4%) |
| 2019 | 8,803 | 7.2% | 79.6% |
| 2020 | 8,781 | 7.4% | 74.4% |
Data Analysis
To analyze CHS data, a weight is applied to each record. The weight consists of an adjustment for the probability of selection (number of adults in each household / number of residential telephone lines), as well as a post-stratification weight. The post-stratification weights are created by weighting each record up to the population of the UHF neighborhood, while taking into account the respondent's age, gender and race. Starting in 2009, responses were also weighted to account for the distribution of the adult population comprising three telephone usage categories (landline only, landline and cell, cell only) using data from the New York City Housing and Vacancy Survey. In 2011, additional demographics, such as number of adults in house hold, presence of children in household, marital status and education attainment, were included in the post-stratification weights. Appropriate calculation of variance and standard errors requires adjustments for the stratification and dual-frame design of the CHS.
If you need assistance with the data, wish to suggest additional variables to be added, or have additional questions about the survey's methodology, please send an email to survey@health.nyc.gov.